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Aw  FPROBLEM TiS rer. 


The Naval Postgraduate School uses a complex prediction 
system to forecast expected student demand for courses over 
the upceming year. This forecast is computed using various 
inputs from several sources during the summer guarter (July 
to September) and covers the following academic year - fall 
through summer quarters inclusive. The tabled output of 
course loading predictions so computed is used by the 
various Department Chairmen to schedule faculty teaching 
responsibilities and research time for that year. As with 
any forecasting system, this set of course loading predici- 
tons is not exact. Various errors are introduced either 
from the simplificaticn of the process reguired to get the 
‘model to a managable size or from erroneous input. The 
Operations Research Department Chairman noticed an inaccu- 
racy proklem when a class with a predicted size of approxi- 
mately 30 was scheduled and approximately 75 students showed 
Dp, causing a disruption of faculty teaching plans. 
Believing that a problem existed, he posted a reguest to 
research the scope of the inaccuracies as pertaining to the 
courses taught by the Operations Research Department. The 
study's goal would be to determine if a Significant problen 
existed and, if so, to provide either a series of adjust- 
ments to be made tc the present predictions or a_ better 
prediction method. 


Be PRESENT SYSTEM 


Until 1972, the scheduling and course prediction systen 


was run by hand using "corporate memory"; the historical 
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demand for courses as recalied by the curriculum representa- 


tives involved. These recollections were used to prepare a 
proposed schedule that the students and curricular offices 
used and modified as required. py 1972, this system had 


just about reached tke limits of its capabilities. At that 
time, the scheduling problem was passed on to a newly hired 
NPS computer program designer! who tailored a predictive 
computer model to the school's needs. That model, with 


modifications, is still in use today. 


1. Scheduling / Frediction Problems 


The problem faced in 1971 is the same today - How 
can we accurately predict three items: 

e The courses required to match student demand 

e When those courses should be taught 

e How many students will attend 
Iwo types of student inputs are available; those who are not 
yet onboard {with or without orders), and those presently 
attending the school in any given capacity. Both tyres of 
students are designated as attending a given curriculum, and 
from that designation a standard program of courses is 
assigned. This assignment leads to the course scheduling 
output. Since the two input groups are significantly 
different with respect to treatment within the model itself, 


they will be separately discussed. 
a. Students not yet onboard 


The Office of the Chief of Naval Operations 
publishes a yearly qucta plan, [Ref. 1], which directs the 
Commander, Naval Military Personnel Command to provide the 
officer personnel to fill the allocated graduate education 


bitlets. The fiscal year 1985 quota summary is shown in 


1Mike Troian Spas 4 Assistant for Programs; presently 
working under Code 013: J. Barron, Gapt. Us (ret) 
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table I . These quotas are further broken down to show, by 
professional designator (1.e. Surface Warfare, N13 OF 
Cryptology .. .), what curricula the officers will attend. 
Historically, however, these quotas have been guidelines and 
not strictlym= forlovwed: Additionally, Naval Military 
Personnel Command (NMEC) gives only a rough idea of in which 
guarter an officer will arrive during a given year. This in 
turn drives the input problem experienced at NPS. Through 
liason with NMPC, the school identifies as specifically as 
possitle just who has orders to attend NPS either in hand or 


being written. Quota shortfalls tend to occur in the "tech- 


NMical" curricula due to a lack of available, qualified 
personnel. Conversly, surpluses occur in the "nen 
technical" curricula. As with any military organization, 


operational commitments such aS maintaining ships in the 
Eastern Mediterranian Sea or Indian Ocean adversly affect 
the arrival of Surface Warfare Officers (who consitute the 
largest group in the NPS student population). 

The school folds all of this gquota information 
and experience into a "best guess" of the number of students 
arriving per curriculum per gquarter.? As this process is 
being completed, the curricular officers designate the rfaths 
these new arrivals will follow. This is done either by 
service (i.e. USN students take the following courses: . . 
-) or by past experience (i.e. 28% of incoming students need 
refresher math followed by . ..). The resulting linking of 
numbers of students and assigned courses provides one type 


of input to the scheduling model. 


ee Ses 


_. “Additionally the programmin office contacts indi- 
vidual service and foreign representatives to further refine 
the expected inputs (since [{Ref. 1] pertains only to _USN 
officers). These organizations provide less reliable data 
mainly because the fiscal year for other US services starts 
in January. The eps ee their personnel transfers is 
therefore not complete until October or November. 
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rk. Students presently attending NPS 


The scheduling process’ for those students 
already onkoard is scmewhat simpler. Since they are already 
designated as attending a given Curriculum over a_ known 
period of time, the cnly area in which significant problems 
are found is that of elective scheduling. Since the 
prediction is completed a full year in advance, out-quarter 
electives may not have been correctly selected by the 
student. His interests or his reguired course load may 
change, thereby changing any electives he has’ selected. 
Additionally, the course demand may be such that the course 
ls withdrawn from the list of available courses. ineaaen 
cases, however, course selection input covering the upcoming 
year is made by the student through his curricular officer 


to the scheduling office. 
2. Scheduling / Prediction System Use 


Once elective selections have been made by the 
onboard students and the numbers of new inputs per curric- 
ulum are entered, the course loading prediction model 
applies standard course matrixes to generate the overall 
student course demand. The scheduling system is demand 
based, meaning this derived demand drives the faculty hiring 
and scheduling process. The budget process also is based on 
the course demand. Teaching funds are allocated for ali 
instructors teaching course sections with five or more 
students. The individual department chairmen receive a 
listing of student demands by course and guarter from which 
they create a proposed schedule of courses to be cffered. 
This schedule is further adjusted through the allocation of 
teaching funds in a frocess consisting of an approximately 
month long dialog between the department chairmen, division 


deans, and administrative representatives from the 
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programming office. The goal of this process is to create an 
acceptakle schedule of courses to be offered talanced 
rFetween annual operating budget funds, student course 
demands and faculty research requirements. Through this 
scheduling process, some courses are dropped and others 
added, creating some student movement among the courses 
offered. The goal of the course loading prediction model is 


to minimize the course changes caused by this scheduling. 3 


3Mike Troian, the Special Assistant for Programs working 
from Ingersol Room 375, _.has routinely conducted accuracy 
reviews of the course loading prediction model. For FY1983 
this review has shown a_ 90% accuracy in predicting actual 
courses taught with aless than 1% deviation from_ total 
number of courses pues cree (his figures). Neean I1ndveator 
of faculty reguired and expected funding costs, the model is 
reportedly excellent. 
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II. INITIAL STEPS i ANAmwSTS 

Before any fact-Ltased discussion of "better" predictions 
could bte started, the problem facing the Department Chairman 
had to te defined. A single historical observation ofa 
severe prediction error that caused problems is not signifi- 
cant considering the number of predictions and observations 
made. Historical data had to be checked to determine if the 
problem cf inaccurate predictions actually existed or was 
merely assumed based on a seldom occurring event. Since 
predictions were made by course, the obvious choice for 
comparison was the historical course data available from the 
registrar's office. Not until these steps were completed 


could a sé€arch for a ktetter method of prediction be started. 


A. DATA ORGANIZATION 


One of the initial steps of the analysis was an organi- 
zation of all the usable data into a yguantifiable fcrmat. 
Useful information available consisted of the following 
items: 

ecourse number 

eguarter taught 

enumber‘of students predicted 

enumber of students attending 
The eventual goal of the analysis became that of finding 
such permanencies in the data (based on these items) as to 


provide an accurate prediction adjustment. 


1. Prediction Data 


_— 





Prediction data was available both for individual curricula 


inputs into a given course and for the total exfected 
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student demand for those courses. This data was presented 
in two formats: summary data and individual course data. 
The summary data (figure 2.1) existed for academic years 
1982, 1983 and the fresent. This data provided a given 
years display of the total expected student loading indexed 
by course and guarter. It included a recommended number of 
“sections taught" and _ the corresponding faculty hours 
required. As seen in the figure, the predictions are made 
for a given academic year, in this case 84 (September 1983 
to August 1984). The first column contains the alpha- 
humeric course identifier as listed in [Ref. 2]. mnie wSiut 
columns refer to the number of students predicted to attend, 
the "SEC' column refers to the number of sections required 
to accomodate those students (maximum section size is 
limited to 30 attendees), and the 'FAC-HRS*' columns refer to 
the number of faculty teaching hours needed to teach those 


secticns. 
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POGwees and 1984, individual course predictions 
were available (figure 2.2). These course predictions listed 
the expected student loading for a given course during a 
given guarter coded by degree awarded. This degree coding 
is found inthe "S/D/T" (specialty/degree/tine) column of 
the data. The first two letters of this column apply to the 
curriculum specialty, the next two to the degree (MS refers 
to Master of Science). This coding is easily transferrable 
eo Curriculum number using Naval Postgraduate School 
mistruction 1520.19 (figure 2.3). The numbers that occa- 
Sionally appear following those four letters indicate when 
curricular officers have not provided an updated list of 
whom they expect for that course at that time in that 
specialty. The numbers mean that the prediction listed came 
from the "standard files" of the programming office; a 
collection of best guesses of the expected number of atten- 
dees loosely based on historical patterns.* These individual 
course predictions, however, make no mention of Segment 


asSignments. 





4Cther columns in the individual course prediction data 
are generally self-exp ea one with the exception of these 
Minor items: The AV column referred to_the number of avia- 
tors (Pilots, NFOs, etc. 2h that attend and is an unused relic 
of some past progran. he letters sometimes preceding the 
section column are now meaningless. The section column code 
itself is Seat ghee by EheCmmGULEICULlar offices to groups of 
incoming studen 
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Example of Individual Course Predictions 


Figure 2.2 


i cee a a I A mm a Rt  s O  t iiiniagg SOTA as cer i i OE iran: Coe ac Nami eine AA unis MN GRIP ae 5 ig A 


NAVPGSCOLINST 1520.19 CH- 
1 October 19 
CODES USED FOR SCHEDULING 
AS OF 
JUNE 1983 
SREGLALT Y 
DEGREE 
CODE 30 CODE SPECIALTY 
360 RAMS O8/SA ARMY 
RGMS OR/SA GENERAL 
RNMS OR/SSA NAVY 
RSMS OR/SA SUPPLY CORPS 
RPPH OR/SSA PHD IN OR 
RCMS GROUP OF ANY OF ABOVE 
CODE es) 
€ 10 CMS AERCNAUTICAL ENGINEERING 
611 AXMS AEFCNAUTICAL ENGINEERING (Avionics) 
ATMS AEKC ENGINEERING FOR TEST PILOTS 
CODE SZ 
600 DCMS ELECTRICAL ENGINEERING (Communications) 
590 EAMS ELECTRICAL ENGINEERING 
620 HCMNS COMMUNICATION MANAGEMENT (Coast Guard) 
HANS COMMUNICATION MANAGEMENT 
DoD KE ome Er eCIRNONLC eyARFARE oY SlEMS 
366 pemo SPACE SYSTEMS OPERATIONS 
51S) SENS SPACE SYSTEMS ENGINEERING 
CODE. 33 
200 GMMS MATH OPTION OF ADVANCED SCIENCE 
381 Coon eteo “OPTERON WOF ADVANCED SCIENCE 
£30 WIMS WEAFONS SYSTEM ENGINEERING 
S31 WSMS WEAFONS SYSTEM SCIENCE 
Bo WNMS NUCLEAR PHYSICS (Weapons and Effects) 
23) UXMS UNDERWATER ACOUSTICS 
GOODE 331 
25 IXMS ANTISUBMARINE WARFARE 
Figure 2.3 Partial Listing of Degree to Curriculum Codes 
(see [Ref. 3] or Appendix B for complete listing). 
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2. Historical Data 


Historical attendance data was made availatle by the 
registrar's office. This data, filed by quarterS listed the 
courses taught in alpha-numeric order displaying student 
names, curricula, and grades received (see figure 2.4). The 
numbers recorded reflect the number of students completing 
the course vice the number seated on the first day of class. 
Segements are listed separatly. Use of this historical data 
implies a number of actions. First, the predictions are 
demand Etased and would probably correlate better with the 
humber of students registered for the course than with the 
humber ccmpleting. A strong positive correlation is assumed 
between number of students registered and number completing 
a given course. Seccnd, grade and name information was 
discarded thereby burying any measure of correlation between 
performance and attrition. Third, segment data was collapsed 
to match individual course prediction data. This opscured 
relationships between predicted and actual number of 
sections taught or faculty hours required. Since the geal of 
the analysis was to adjust the predictions of student demand 
vice to explain the reasons an adjustment 1S necessary, 


these actions were considered acceptable. 


SThe registrar's system 15 rather unique an eepecmee 
Ob thaseedata- "Schcol Year 84" does not refer to fisca 
ear 84 (normally 1 October 1983 - 30 September 1 
nstead "School Year 84' refers to the September 84 to 
August §5 quarters. As this data was coliected, the codin 
of quarter data (tc be discussed later) was translate 
back intc standard fiscal year notation. Thus, _ throughout 
the data base used in the analysis, 83.1 refers to’ the 
September 82 - December 82 quarter of the 1982 - 1983 school 
year, as expected. 


Ze 


| OFFICIAL GRADE ROSTER PAGE 1 | 
| 
NAVAL POSTGRADUATE SCHOOL | 
MONTEREY,CALIFORNIA 93943 DATE 13.58 PR [ 
| COURSE NO. OS3602 SEGEMENT 00 --- QUARTER 1 AY 83-84 | 
| 
COMBAT MODELS PROF. SHUDDER, REX H. | 
[ 
i 3 LECTURE HCURS 1 LABORATORY HOURS | 
| 
CORPS/ 
NAME RANK COUNTRY CURR MARK | 
1 Kx Kee KEK Xx Eo USN #374 A 
Zz. MEX XMXE XXXX X LEwR “USN #374 B | 
See XXX, OX KK x LCDR USN #525 B l 
| oe XXXRXK, XXXX X 1058 USN #525 B { 
SemexK Xu, XXXX XxX LCDR USN #525 A- | 
Gee XXXXKX, XXXEX X lah USN #525 A 
Ua. KBEXXKX, XXXX X LCDR USN #374 A= | 
eee KNX, XKXX X ES USN #525 A 
| oe) Pe XXXR XK, XKKXX X Ly USN #525 B | 
10s) RRXXXX, XMKX X ce USN #525 B 
im) la RXEMXX, XXXX .*X LCDR USN #525 B l 
eee Xe, OX XXX cy USN #525 B | 
| FORM 4 | 
[ 
ios to THE OFEICTAL GRADE .ROSTER TO BE FILED IN THE J 
ACATEMIC DEPARTMENT | 
= { 
— J 


Figure 2.4 Sample Attendance Data (names removed) 


3. Coding System 


Before entering the data into computer files, a 


coding system was devised to distill the raw data into its 


usable parts. Excess 
grades could then Ee 
predicticns and actual 


tLELeCTS. Tdentifiers 


information such as student names and 
discarded. The data consisted of 
attendance figures coupled with iden- 


reguired include course number = and 


quarter taught. Available data included both total and indi- 


vidual curricula figures. For simplicity of use, a kasic 


file was created using the following five-column coding 


scheme: 


Ze 


Quarter Curricula# Course# #fattended #predicted 


Once these items had been tabulated, an easy subtraction 


(attending minus predicted) provided an error. tern. By 
ordering (by magnititude of error) and displaying the 
resulting array, a history of many poor predictions was 


cbserved. This observation confirmed that a problem existed 


and showed a need for further analysis. 


B. NATURAL DIVISIONS IN CODED DATA 


The large volume of coded data created was divisible 
into several sub-categories. The Operations Research 


Department teaches twe types of courses: OA - taken by oper- 


ations analysis major students; and OS - taken by students 
from cther curricula. Initially both of these types were 
taken under consideration for analysis. Later, the OA 


designated courses were dropped from consideration for two 
reasons; the problem of not knowing how many students were 
going to show up was generally solvable within the depart- 
ment; additionally, time constraints limited the score of 
the areas to be covered.® The OS course data was broken into 
two prediction areas: summary predictions and curricula 
specific predictions. The summary predictions were used to 
show patterns by course; the curricula predictions to show 


patterns by individual curricular offices. 


1. Summary Data 


When dealing with summary prediction data error 
terms, the goal of the analysis was to provide a functional 


relationship between predictions and attendance ky course. 


a SS ce se ee a ee 


6CA data collected ees the ae ee stages of problen 


lnvestigation is availab e> scrnat 1S) sii gm ei 
different from that of the OS course data. because’ the 
robliem found for OA dealt with elective selection. The 
actcr of interest was service affiliation. FOr fuEthner 


details, see appendix A. 
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Using these derived relationships (hopefully simple addi- 
tions or subtractions), the department chairman will be able 
to take the forecast from the programming office (figure 
2.1) and change the figures shown to reflect more accurately 
the expected student load. These more accurate figures can 
then be used to schedule faculty teaching assignments, 
request necessary funding, and accomplish other tasks 


requiring the number of expected students per course. 


2. Curricula Specific Data 


When dealing with error terms for individual 
curricula, the goal of the analysis was to provide a func- 
tional relationship Lretween predictions and attendance by 
curriculum. Using these derived relationships, the depart- 
ment chairman will be able to take the forecast for an indi- 
vidual course (figure 2.2) and change the figures shcwn to 
reflect more accurately the expected student load by either 
applying the relaticnship or calling the curricular office 
involved to get better numbers. These historical observa- 
tions will enable the department chairman both to identify 
curricula that are inadvertently causing scheduling protlems 
and te show improvements or deterioration in the scheduling 
process itself. This information can be used both by the 
programming office and the curricular office involved as 


well as the OR Department. 


Zo 


III. DETAILED ANALYSIS OF OS PREDICTIVE DATA 

At the completion of the initial steps toward analysis 
previously discussed, a coded series of data arrays were 
available for use. An example of array formatting is shown 
in figure 3.4 The figure of interest was the "error" tern 
found in column six. These error terms, representing the 
difference between the actual number of students attending a 
course vs the number predicted, were displayed in various 
groups ordered by magnitude to show the accuracy patterns of 
the forecast. Areas that were explored for effects included 
the curriculum that frovided the input, the courses then- 
selves, and the guarter for which the courses were 
predicted. Underlying patterns such as an individuals 
“"nDath" through various OS courses reguired for his curric- 
ulum were also investigated. The paucity of the data 
precluded the use of sophisticated statistical techniques 
thereby limiting results to indications of areas of concern 


rather than definitive statistical statements. 


A. THE DATA 


Six element data arrays (as shown in figure 3.1) were 
created for each years data. These APL variables were 
combined to form a single compilation of all the available 
data (see table ITI). The error terms of this compilation 
were used for the analysis. These error terms are discrete 
integer data the size of which are limited by the maximum of 
the actual or predicted columns terms. Distributional 
fitting was not done for two reasons; the population should 
have high internal correlation due to the interrelation 


rFetween course errors caused by student course matrices, and 
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CTR CURRIC CSE ACT PRE (ACT-PRE) 
KK HK KKK IK KOK KK KK XK EK A KK KK 
84. 3 02103 0 1 -1 
BulS S355. 2603 0 1 -1 
84. 3 C3001 25 0 DE 
84.3 368 3091 25 0 25 
84.3 0 3002 15 18 -3 
84.3 683 3002 0 1 -7 
e6.3 825 #3002 15 #17 -2 
83.1 0 3602 7 7 0 
83.1 374 3602 1 1 0 
83.1 525 3602 6 6 0 
le | Oeecos 20° «(21 -17 
Cem 595) 60S 20: 21 -4 
83.1 0 3604 18 #16 2 
83.1 365 3604 1 0 1 


OTR: Cuarter taught. 83.1 refers to the first quarter 
in the Sept. 82 to Aug. 83 school year. 


Curric: Curricula - 0 implys summary data for a given 
course. 


Cse: Course number (all OS courses). 


Act: Actual number of students listed on final grade 
records as ccmpleting course. 


Pre: Predicted number of students (from forecast). 


Act-Pre: Prediction error tern. 


a ESS WA a A ag = Ah Mp gy A ge a a ya gS ey ey Us GMMR ee ei es I nem) Ms 
erate oe Ga OE pe ee Go eee Siete eet eatin Ee qty GEIS Gemstone RD «ey Ss SE eee gece I cei et ee ee ec BR ee oe 


Figure 3.1 ee of Array Format (with definitions) 
APL variable used : OS. 


the sample sizes of those homogeneous populations distilled 
from the data (Such as "all 365 students taking OS 3604 in 
the second quarter") were prohibitively small. Two tyres of 
errors were discovered, predictive errors and accuracy 
errors. A predictive error was defined as either the fore- 
cast predicting no students and students attending or vice 
versa. Embedded in this error type are those courses which 


were not even listed but taught and those courses dropped 
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cern ace ages eects err cme USAR meee pemeines oeeerin SSRIS cs eee cee eee (B28 [Lo fo oe eee es EE 


TABLE If 


APL Variables used in Analysis 
(see Appendix A for details of use) 


(without repetition) 


VARIABLE SHAPE SRESCRIPTION 

ok RRR ROK kkk kokok ok ok kok kok kok ok ok kx 

OS52 69x6 AY (Academic year) 81-82 data; 
| includes Sune nt Ofna oe 
| (no data for individual curricula) 
| 0S83 2256 AY 82-83 data; complete listing 
{ OSTRKE3S 140x8 AY 82-83 non-entry course data * 
| Oosé4 228x6 AY 83-84 datagmonly@qucs FZ anda 
| OS 39 2X0 Ccmbination of all of above 


* see 'Course/Curriculum Interactions and Tests' 
section for complete description. 





from the schedule in the budgeting process. The same divi- 
Sion was found by individual curricula (for example, curric- 
ulum 460 (engineering science - preparatory) is never 
predicted in any individual course forecast). Predictive 
errors exist when cclumns five or six of the data array 
contain a zero. Accuracy errors are other differences 


between the actual and predicted number of students. 


B. CCURSE/CURRICULUB INTERACTIONS AND TESTS 


Each individual student at the Naval Postgraduate School 
follows an assigned ccurse matrix. This asSignment varies 
by both the program assigned and individual student needs. 
However, all matrices list aoseries of courses that are 
either required or elecive. The Operations Analysis curric- 
ulum, through its OS course offerings, provides a service to 


other programs by previding basic and advanced pobability 
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PiiGmoredtasties ~training not available in other curricula. 
These service courses are generally reguired by cther 


curricula as a frerequisite to both further training and 


degree completion. As such, they are taught in a certain 
sequence. The amount of training required differs with 
different programs, So various OS "tracks" are reguired. 


Sequencing of courses in this manner will cause dependence 
between the error terms of certain courses. To determine 
the dependancy relationship within the data, the course 
Matrices for all curricula offered at the schocl were 
collected andthe O£ course seguences extracted. These 
sequences were combined to form a series of condensed course 


matrices as applicable to the Operations Research Department 


(see figure 3.2 - Note: x's refer to students onboard but 
not taking OS or OA designated courses). From these 
matrices, patterns emerged. As with any program, students 


Started with basic courses and moved to more advanced 


TOpaCcS. Five basic courses were defined as “entry level" 
courses. One of these courses was required before any 
advanced work was done. Since the entry level courses were 


generally taken early ina student's time in his frecgran, 
and since the entry course was the first time he appeared 
eimegible for “*prediction", the entry level courses were 
expected to have the highest variance in error terms. This 
proved to be true. As the students spend more time in their 
programs, attrition and schedule changing should level off. 
Additionally, since most students spend more than a single 
year going through their programs, "“down-line" predictions 
will be improved merely by the students moving from the "not 
yet onboard" to the more stable “attending NPS" prediciton 


group. 
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Definition of entry level courses led to the definition 
of course seguence "tracks", A student from a given 
curricula will take an "entry" course - the first time he is 
seen by the OA Department - and will then take follow-on 


courses EFased on his course matrix. BACH CUELICWlUMN nas Lts 


Pra ae | 
ENTRY TRACK Ee be 
CCURSE CURRICULUM COURSE AL 
HARKER RKO KKK LKR RK KKEAK 
0$2103 365 3604,3404,3008, 
| 3603,4602 
i 366 3604,3404,3008, { 
{ S603 ,24602 ; 
374 3601, 3602,3608- | 
| 3604 | 
525 330 3-500 soul. 
3401,4601,3602 
535 none | 
595 S60G7 S005 7506003, 
| a0 5 601 | 
| 0$3101 S67 3004 | 
i 620 3005 { 
686 none | 
| 687 none | 
825 3404,3002 | 


; 


Figure 3.3 Example of Entry vs Track 
Course Organization. 


own "track" (See figure 3.3). Using these tracks, it should 
have been possible to derive deterministically the exact 
course loading for those follow on or track courses given 
the course matrixes and attendance figures for the entry 
level courses. Using the FY83 data, this was done. The 


resulting matrix was set up in the following manner: 
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Quarter Curric Cse Act Pre Dpre (Act—Pre) (Ace-peee) 
Act : Actual number attending 
Pre ;: Standard predictions 
Dpre : Lerived predictions 


A display of the results showed no observable difference 
between the regular error term and the derived error tern. 
A Wilcoxian signed rank test? was done to test the 
hypothesis that the error magnitude of the adjusted 
predictions tends tc be smaller than the magnitude of the 


error frem the forecast (see table III). 


. 
= 


TABLE IiL 
Wilcoxian Signed Rank Test Results 


Sensitive assumpticn; Differences independent. 
Assumed large amcunts of varied data negated 
track effects resulting in relatively independent 

ifference terms. 


CASE ASE B 
Variatles Tested: xX =Curric ACT-DPRE Sum ACT-DPRE 
=Curric ACT-PRE Sum ACT—PRE 
H The X values do not tend to be smaller than 
0 the Y values. 
x 
Y 


Ou 
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ee 
P Og 
On 
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comes 
os 
on 
Sa) 
ve) 
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toe 
op) 


H : The 
4 the 


Decision Rule : Reject at the 5% confidence 
evel if TSeoek 


a CASE A CASE _B 
Test Statistic : : =1.455 


Conclusion : Cannot reject null hypothesis. 


values tend to be smaller than 
values. 
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The results of the test indicated that the adjusted 
predicticns were ne improvement over the forecast. That 
concliusicn indicates that interactions more complex than the 
following of course matrixes are involved in actual student 
movement from course to course. The adjusted prediction 
method was dropped from the analysis as a means of better 


predicting course loading. 


C. CTEER INTERACTIONS AND TESTS 


Interactions exist in areas separate from the dependen- 
cies created within a class seguence by the course matrix 
structure. Three main effects were studied. FIEst, 
curricula may have individual patterns across time. Scne 
groups may consistently have an optimistic input that causes 
an overestimation of number of students wherever those 
groups appear. Second, individual courses may show patterns 
of over- or underestimation. Lastly, a quarterly effect on 
errors May appear due strictly to the elapsed time between 
the prediction and the actual class. To provide an initial 
look at the data to show such patterns, two programs were 
written.® These programs were designed to sort the data into 
the catagories of interest, specifically curriculum, course, 
and guarter. The resulting groupings showed that there were 
patterns in each catagory, but that the number of observa- 
tions available to base conclusions upon made those fatterns 
Statistically insignificant. Through clever data selection, 
a series of observations was gleaned to test for curriculun 
and course effects using median polish technigue as detailed 


in {Ref. 4}. The results of that test were inconclusive 


Sa Se SE cmc ee ee 


8These programs and others used in the analysis are 
fully explained and documented in appendix A. That appendix 
also ccntains instructions for the maintenance and use of 
the data base created, and procedures for gaining access to 
same. 
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showing no temporal stability (see table IV). The test for 
guarterly effects was not done due to the paucity of the 


data. ? 


3Complete data existed only for FY83. This limited 
ae was evaluated as too Unhrepresentative (6 be Seat tsrere 
cally significant concerning quarterly effects acrcss 
various yé€ars. 
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IV. CONCLUSIONS 

The problem as defined was to aid the Department 
Chairman in his task of scheduling and budgeting for faculty 
teaching responsibiiities by providing either ae better 
prediction method or adjustments to the present forecasts. 
From the scheduler's standpoint, jumps across any nuitirle 


of thirty students cause new sections to be created or 


dropped, and courses with a demand o£ less than five 
students are dropped. To the scheduler or budget office, 
these are significant numbers. The Department Chairman, 


however, cares little about the actual number but much about 
the accuracy of the prediction. Since predictions are made 
by course and broken down within individual courses by 
Cube LCulun:; adjustments were found for courses and 
Cisne) a. Predicticn data were the limiting quantities; 
predictions by curricula exist only as far back as FY83. 
This limited the scofe of the investigation to 1 and 3/4 
years. With this limited history, indications of problem or 
non-problem areas are the only adjustment/accuracy measures 


possitle. 


A. CCNCIUSIONS BY COURSE 


1. Program Qutput Utilization and Displays 


Indications of predictive accuracy by course were 
obtained using the ‘CCURSEP* and ‘COURSE programs detailed 
in Appendix A. The 'COURSEP*" (standing for course patterns) 
program was used ‘first. This program takes the input 
Matrix, in this case the variable 'OS' containing all of the 
data so far available, and displays it grouped by course in 


decreasing order of predictive error magnitude (see figure 
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| COURSE : 3602 ; 
| -82. 3 0 2602 7 11 = 
| -83.3 374 3602 3 5 -2 

-84.3 374 3602 ? i -2 

=84.3 525 3602 12 10 2 

~82.3 0 3602 15 14 1 

-83.3 0 260 2 15 16 = 

-83.3 686 3602 1 0 1 
-83.17 0 3602 7 7 0 | 
| -83.1 374 3602 1 1 0 

-83.1 525 3602 6 6 0 | 
| -83.3 525 3602 17 11 0 | 
=34. 1 0 3602 12 12 0 

-84.1 374 3602 3 3 0 

-84.7 525 3602 9 9 0 
| -84.3 0 3602 14 14 0 


a, 


Figure 4.1 Sample Output from COURSEP Progran 


4.1 for a single course example) .!° This output is quickly 
scanned for ‘interesting' courses ~- those with either 
Significant problems or very snall, infreguent errors. 
Those courses with protlems are further checked to determine 
if there is any pattern to the errors observed. Many 
negative numbers will indicate an overestimation prcblen 
(prediction greater than actual attending), conversely fposi- 
tive numbers indicate underestimation. Quarters may be 
split indicating large errors for some, small for others. 
Curricula may be grouped together indicating prediction 
performance differences. In the example shown (Figure 4.1), 
€rror magnitude is sgall and signs vary, indicating rela- 
tively minor deviations from the predicted values. Had 


these errors been large, the following observations could 


10Tn this display (and all others), the quarter is shown 
as a hegative number. This is done strictly asa formattin 
adem on API a space is added to each column in the outpu 
eee ce to allow ne Cami eS) Cio Sie, CLLOL terms OCCaslon-— 
ally ad no negatives. When this occurred, the output 
display deleted the six spaces ‘saved' for the_sign. This 
caused an irritating misalignment in the columns of the 
a ae The easy correction was to ensure a negative 
existed in all cases, thus the quarter coding. 


a) 


have keen made: 
e Third guarter errors tend to be higher than 


first quarter errors. 


e No curricula appears to do any better or worse 
than any other, though 374 may tend to over- 


estimate in their predictions. 


To further detail predictive behavior within a 
course, the ‘COURSE program is utilized. This program 
takes the same matrix previously investigated (in this case 
‘OS') and also displays i1t grouped by course, but with 
various catagorical presentations made. Output may be 


displayed in quarterly or curricula subcategories (through 


answering program frompts). With «the curricula veetian 
selected, the output compares patterns of individual 
curricula (see figure 4.2). Statistics displayed include 


the mean and variance of the error terms for individuals 
curricula, and predictive, accuracy and no error percentages 
also Ey curricula. To avoid counting data twice, summary 
data entries (column 2 = 0) are disregarded. FLOM Scangor 
the example (Zigure 4.2), the following conclusions can be 
drawn: 
e Curriculum 525 provides the majority of the 
input for 083602, and therm predictions 
are excellent indicating no problem with 


the course. 


e Curriculun 374 may show a pattern of over- 


estimation. 


e Curriculum 686 (with 100% prediction errors) 
normally will not provide inputs to this 


course. 
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| a 
COURSE : 3602 
-82.1 0 3602 7 11 -—4 | 
=—58 7.3 @) 3602 15 14 1 | 
-83.3 0 3602 15 16 ={ 
-83.1 0 2602 7 7 0 
-84.1 0 3602 2 A2 0 | 
-84.3 0 3602 14 14 0 | 
-83.3 374 3602 3 5 -2 | 
-84.3 374 3602 2 4 -2 | 
| -83.1 S74 32602 1 1 0 | 
| -84.14 374 3602 S 3 0) | 
-84.3 525 3602 a2 10 2. 
-83.1 525 32602 6 6 Q 
-83.3 525 EGO 11 11 0 
| -&4.1 525 3602 9 3 0 | 
-82.3 686 3602 1 0 1 
| GROUP MEAN VARIANCE N (OF ERROR TER) 

By WO -1.0 ie 3 as 6, | 
225.0 eS 1.0 4.0 j 
! 686.0 1.0 jiaee, 10 J 
OVERALL: —-.1 126 9.0 | 

ERRCK PERCENTAGES 
| GROUP § PRE ACC NONE N ) 

3740.00 ~U0 Misi 6, 5/6, E200 | 
B25. 00 -00 a Pee 4.00 

| 686.00 1.00 00 E00 1.00 | 

| 

| 

(eens Mae. =) 


Figure 4.2 Sample Output from COURSE Progran 
Options ;: Curricula Grouping with Error Percentages. 


The curricula option of the 'COURSE' program aprfears 
to be the most valuable display available. The information 
Shown is directly transferrable to the prediction data as 
presented (refer back to figures 2.1 and 2.2). 

With the quarter option selected, the output 
combines all of the data into four catagories depending on 
the guarter in which the course has been taught (figure 
4.3). The same statistics are displayed, this time for each 
quarter. Again, Summary data entries are disregarded to 
averd dwelacation of data. From a scan of this figure, the 


following conclusions can be drawn: 


ae 


ERROR PERCENTAGES 

ROU PR AC 

@ Ld 0 2 0 
S200 : 


j= 
- 
Nd 
OO 


= 
be 
Waa 


OO 
OO 
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| 
| COURSE : 3602 ; | 
-82.1 0 3602 7 11 -4 
-83.1 0 3602 7 7 0 | 
-83.1 374 3602 1 1 0 
~83.1 525 2602 6 6 0 
~84.1 0 3602 12 12 0 
-84.1 374 3602 3 3 0 
| -84.1 525 3602 9 9 0 | 
| -83.3 374 3602 3 5 -2 | 
-84.3 374 3602 2 i -2 
-84.3 525 2602 12 10 2 | 
-82.3 0 3602 15 14 1 
- 83.3 0 3602 15 16 =] | 
-83°3 686 3602 1 0 1 
-83.3 525 3602 11 17 0 
-84.3 0 2602 14 14 0 | 
| GRCUE MEAN VARIANCE N (OF ERROR TERM) 

1.0 z0 0 4.0 
| 350 -.2 3.2 5.0 | 
| OVERALL: -.1 1.6 9.0 | 
| GROUP PRE ACC | 
| 1-00 0 ye | 


Figure 4.3 Sample Output from COURSE Prograg 
Cptions : Quarter Grouping with Error Percentages. 


e 083602 has only been taught during the fall 


and spring quarters. 


e Fall guarter predictions tend to be exact. 


Spring quarter predictions are worse, but not 


to such an extent as to cause problems. 


Using the previously discussed displays, the courses 
were separated into three catagories of interest: those 


with indications of significant prediction accuracy 
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problems, those with relatively minor problems if any, and 


Ewoce With too Tittle information to draw conclusions (see 


; 


3 7 


TABLE V | 

Course Results | 

Prceblem Courses "Good! Courses | 

yoy 2302 2103 3002 3004 3006 | 

S008 3008 3101 3303 3402 3601 

S402 3103 3108 BO02°~ 3603 > 3608 i 

See S4Oh 3702 4601 4701 

| 

| Inadequate Data | 
| SOOse 3005 3302 

3401 3403 4602 | 

table V). Predictions made for courses can be checked 


against this listing for an indication of their expected 
performance. Should more detailed information be needed, 
Appendix C contains the results based on summary data fron 
81-82, complete 82-83 data, and fall, winter and spring 


guarters from the 83-84 school year. 


Be. CONCLUSIONS BY CURRICULA 


1. Program Qutput Utilization and Displays 


indicaticns of predictive accuracy by curricula were 
obtained using the "CURRICP* and 'CURRIC* programs detailed 
in Appendix A. thes’ CURRiCP* (standing for curricula 
patterns) program was used first. Titoaeprogran Laxes tne 
input matrix, ‘OS* as before, and displays it grouped by 
curricula in decreasing order of predictive error magnitude 


(see figure 4.4 for a single curriculum example). This 
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| 
| 
! 
r 


( 
| CURRICULUM : 374 { 
—-83.4 374 z101 0 6 -6 | 
j -83.4 374 2103 2 6 —4 | 
i —-83.2 374 2103 1 4 - 3 | 
—~-83.3 374 3604 1 Uy -3 | 
-83.4 374 3601 2 5 -3 | 
-83.3 378 3602 3 5 2 
-83.4 374 3603 4 6 =3 
-84.1 374 =604 2 uy -? 
-84, 3 374 3602 , uy -? 
-83.2 374 3601 a) S —1 
-83:2 374 3604 0 1 = | 
| —-84.1 374 2103 0 1 -4 | 
ie ee 2 3401 0 1 a | 
—~8uU.2? 374 2103 2 3 aa | 
—8u.2 "San 3601 3 mn a 
-~84.3 374 3604 2 3 -1 | 
—-83.1 374 3602 1 1 0 
-2@3.1 374 3604 uy 4 0 
-84.1 374 3602 3 3 0 
—-84.1 374 2603 1 1 0 | 
Ly _..§ 


Figure 4.4 Sazple Output from CURRICP Program 


output is quickly scanned for ‘interesting' courses - those 
with -significant prceblems or small, infrequent errors. 
Those curricula with significant errors are further checked 
for discernable patterns such as regularly occuring over- or 
underestimation, or guarterly trends. In the example shown 
(figure 4.4), problems appear due to consistent overestima- 
tion. However, given the magnitude of the errors, these nay 
be viewed as insignificant. It also appears that fall and 
winter quarters tend to have fewer or smaller prediction 
errors. 

To more closely investigate patterns shown, the 
"COURRIC* program is utilized. This program takes the same 
input matrix and disflays it further grouped by quarterly or 
course catagories. With the quarter option selected, the 
output ccmpares predictions across time (see figure 4.5). 
From the example shown, the following observations can be 


made: 
e Overestimation errors do increase across 
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consecutive cguarters. 


e Lower level courses tend to have larger mag- 


than more advanced courses. 


nitude errors 


e 082103 and 0S3401 might not be taken by cur- 


students fall quarter (more 


ELculwum 374 


data required) 
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With the course option selected, information can be 
gleaned on curriculum 374's prediction performance fcr any 
given course (see figure 4.6). Again from the example 
shown, conclusions can be drawn: 

e 082101 and 0S3401 are generally not taken by 


students front Curriculun’ 374 


e Larger errors cccur in 0S2103 and OS3604 (as 
expected from the course matrix - refer back 


to figure 3.2) 


2. Display Based 


Using the Freviously discussed displays, the 
curricula were separated into three catagories of interest: 
those with indications of significant prediction accuracy 
rroblems, those with relatively minor problems if any, and 
those with too little information to draw conclusions (see 
table VI). Predicticns made for courses can be checked 
against this listing for an indication of their exfected 
performance. Should more detailed informtion be needed, 
Appendix C contains the results based on summary data from 
81-82, complete 82-83 data, and fall, winter and spring 


quarters from the 83-84 school year. 
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APPENDIX A 
PROGRAMS AND DATA BASE USERS MANUAL 


This appendix contains the detailed information reguired 
to effectively run and maintain the programs and data base 
used for historical validation of the Naval Postgraduate 
School course loading forecasts. The purpose of this series 
of programs is to conveniently display historical course 
attendance data versus predicted attendance data. These 
displays can then be translated into simple adjustments to 
ke made to the forecast, providing a more accurate figure to 


ke used in the faculty teaching assignment process. 


A. ACCESS PROCEDURES 


The data and manipulative programs associated are held 
on the IBM 3033 mainframe at the Naval Postgraduate School 
under user number 1502ZP. The are contained in an APL work- 
Space called ‘OSDATA*. Assuming you have a valid user 
account on the main NFS computer system, this workspace can 
ke accessed in three steps; linking to user 1502P, accessing 
his disk, and copying the workspace onto your own disk. 


Once you are ‘logged on' to the system, the command 
SP elapnk la02p. 195 195 rr 


will cause a prompt to appear asking for the ‘read‘' pass- 
Wand. @hmter.'ps3000'. That password allows access to user 
1502P's disk. To get the workspace transferred to your cwn 


disk, enter 


ace 1959074 followed by 


copy osdata vsarlws b==a 


4] 


B. RUN FPREPARATION INSTRUCTIONS 


To effectively run the programs available, data must exist 


in a six column numerical matrix format coded as shown: 


Column Sample 
Number Entries Comments 
1 Sores | This column corresponds to the guarter 
84.4 from which the data is gleaned. '83.1' 
corresponds to the Fall gquarter of the 
1982-1983 school year (as in standard 
fiscal year notation). Similarly, 
84.4 corresponds to the summer guarter 


of the 1983-1984 school year. 


2 0 This column holds the curriculum code. 
374 'O' means that the entries to follow 
are total number of students predicted 
and tctal actually enrolled in the 
listed course as compared to ‘374! 
which means that the figures to follow 


apply only“to curriculuneses 


3 ya \i0) 3 This column holds the OS course number 
3602 
4 29 This column holds the actual number of 
0 students attending the course as de- 


scrited by the previous columns. If 
column two 1S zero, this number is the 
total number of students that attended 
and should correspond to the total 
listed on the grade record sheet held 
by the registrar. If this number 
itself is zero, the course was not 
taught during the quarter shown in 


column one. 
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D 0 This column holds the predicted number 
Ss of students that should attend the 
course described. If column two is 
zero, this number is the total number 
of students that were predicted for 
the ccurse and should correspond to 
the totals listed on both the summary 
and individual course forecasts. If 
this number itself is zero, either the 
course was predicted to be empty or 


the course was not predicted. 


6 =a This column holds the difference 
between the actual number of students 
attending the course and the number 
predicted (actual minus predicted). 
Negative numbers imply overestimates, 


positive numbers imply underestimates. 


Prior to any program runs, the goal of your analysis 
must ke defined. The programs are generally divided into 
two areas, course centered and curricula centered. In the 


course centered area, the 'COURSEP' program allows a general 


viewing of the error data for chosen courses. This prcgran 
displays the data separated into courses ordered by 
decreasing magnitude of the error term (actual minus 


predicted; column six) and is used to check quickly the 
prediction accuracy Ey course. Then to check for underlying 


patterns by guarter taught or by individual curricula, the 


‘COURSE* program can te used. SPUtlarly, ean the CUuEE:CuLa 
centered area, the ‘CURRICP* program allows a general 
viewing of the error data separated into individual 


curricula, and the ‘CURRIC' program can be used to further 


distill the informaticn available. 


49 


In summary, two steps must be taken prior to starting 


program runs: 
e Ensure the data exists in the proper format 


¢ Decide what infcrmation is needed (define goals) 


C. SAMEFLE PROGRAM RONS 


To demonstrate effectively the varied uses of the 
programs available, a hypothetical problem will be created 
and fully explored. Each program will be discussed in 
detail as it appears in the problem solution, and both the 


inputs reguired and output generated will be shown. 


1. Eroblem Statement 


SSS SS SS SS 


Curriculum 525, Antisubmarine Warfare, and curric- 
ulum 374, Air-Ocean Tactical Environmental Support, are 
going to combine parts of their programs to create a new 
curriculum: Curriculum 449.5 - Antiwhale Warfare. The 
condensed course matrix for curriculum 449.5 has 0OS2103, 
CS3604, OS3601, and OS3602 in guarters 2, 3, 4 and 5 respec- 
tively. As the Operations Analysis Department Chairman, you 
know that 0S2103 and 0OS3604 can handle the additional 
students with no preblems due to the present size and 
frequency of those course offerings. Your guestion, with 
scheduling coming up for the next fiscal year, is whether 
the additional 12 students you expect per 'class' wiil be 
enough to require the offering of an additional section for 
OSSGU Tors GS3602- The figure of twelve students comes from 
the Air-Ccean curricular office, since curriculum 449.5 will 
ke taught as an adjunct of curriculum 374. These students 


will be entering in the Fall and spring quarters. 
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2. Lefinition of Goals 


The first step in the froblem solution is to define 
the information required to answer the question. In this 
case, we need to know if an additional section is required 
given an influx of twelve additional students. This problen 


defines three guesticns: 


© How many students usually take the course in 


question 
e How accurate are those figures 


e How good is the estimate of twelve additional 
students 
Since the first items to be considered involve 
course data, the course related programs will be used. To 
check later the accuracy of the figure twelve, a curricular 
comparison can be made with curriculum 374 (since that 
office is the administrator of the program) using the 
curricula related programs. Additionally, the data used 


must be the most recent and complete information available. 
3. Course Related Work 


The first step we will take is to get a look at the 
data of interest. The variable ‘'0OS*' contains the informa- 
tion we want. The 'COURSEP' program most easily shows this 
dakbea . Through proper inputs, we can call up the historical 
performance of 0S3601 and 0S3602. The ‘'COURSEP*' program 
provides all the prcmpts necessary (see figure A.1). The 
program output is shown in figure A.2. See figure A.3 for 
a program listing. 

As we can see from figure A.2, the actual number of 
students that attend each course appears to be somewhere 
around 15.2 The predictions are presently perfectly adequate 


since they show a reguirement for only one section in all 


5) | 


cases, which holds true. With the addition of twelve 
students, however, the number of students will be close to 
30, necessitating ancther section. To more adequately show 
the data, we can lcok at the ‘COURSE! program sorted by 
GW bake Wile This will break out the summary predictions to 
ketter show how well the course is predicted. Figure A.4 
shows the inputs, figure A.5 the outputs, and figure A.6 the 
program itself. 

Using only the summary data shown in figure A.5, we 
can confirm that the actual number of students that attend 
each course is about 15, and the present one _ section 
requirement holds true. We can further observe a possible 
pattern Ey guarter (errors appear to increase for the fourth 
guarter predictions as compared to predictions made for 
earlier guarters). To more adequately show this pattern in 
the data, we can look at the ‘COURSE program sorted by 
guarter. This will break out the predictions to better show 
quarterly patterns. Figure A.7 shows the inputs, figure A.8 
the outputs. 

In suM@mary, through use of the course related 
programs we have found that a course size of approximately 
fifteen is usual for these courses. Predictions, especially 
those made for the fourth quarter, tend to overestimate the 
humber of students expected. If the figure of twelve addi- 
tional students is in fact correct, a new section will freb- 
ably not be reguired even though predicted attendance values 
are in the twenties without the addition of the new 


CURE Tcu lun. 


5) 


| 
| 





COURSEP QS 


3001 2602 


* see COURSEP listing for details 
of the prompts 


© 
+ 
| Cay ep a 


a 


Figure A.1 Input Syntax : COURSEP 


53 





COURSE * 3601 
-83.4 0 3601 14 2 mS 
G54 WEGGs 3401 3 11 -g 
82). 0 3601 14 24 7 
S822 0 3601 14 g 6 
=G564 wo 3601 2 S ae 
-33.2 0 3401 14 14 9 
"83.4 525 3601 9 7 A. 
"94-28 S65 3601 2 0 2 | 
"84.2 525 3601 20 ae =o 
95.2) Me 34601 4 S = 
"33.2 525 3601 10 fie “4 
=Oane 0 3401 25 26 1 
“Same 374 3601 3 4 =i 
COURSE $ 3402 

| “Sead 0 3602 7 11 “4 
Soe nh 3402 3 S ao) 
SEV eye 3602 A 4 ao) 
"84.3 525 3402 12 10 zl 
=92), 0 3602 15 14 1 
-83.3 0 3402 15 14 =H 
“93.3 686 3602 1 0 1 
“83.1 0 3402 7 7 0 
“63.1 ‘374 zy (oye 1 1 0 
“83.1 525 3602 6 é 0 
"83.3 525 3602 il 11 0 : 
oat 0 3402 1 i 0 
“94.4 37a 3602 Z 3 0 
“84.1 525 3602 9 9 0 
FA 6 0 3602 14 14 0 


Pe 


Figure A.2 Output from Input Sequence shown in 
Figure A.1 ; COURSEP Progran. 
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E22 

E32 

C4) 

ta 

C46] 

C7) 

C8) 

E93 

C10) 
Bol sa 
Er2) 
E13) 
C14] 
Gia] 
Ele 3 
B17 
E18) 
EL94 
C20] 
Faia 
E223 
E222 
C24) 
eJd 
C26) 
G27) 
C28) 
fay 
C30] 
ba. 
ESed 
baa 
C34] 
C35] 
C36] 
C37) 


C38] 
ES7.3 
C40] 


C41) 


TS 


VCOURSEPL 


9 COURSEP 


THIS FUE 


QTR 


COURS 


(ALL QUAR 


TO COMPL 


THE PROGR 
LOOKED AT 


2>r7wFesr?rFe373-ertererererDsverrFrarerteFeraerFrr ? 2 


019 


MATRIXSMsrpyrez 
TIOM, GIVEN AH INPUT MATRIX IN THE FOLLOWIHG FORM } 


* 


CURRIC ese ACTUAL PREDICTED C(ACTUAL-PREDICTED) 


WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 
FOLLOWING MANNER 3 


E 
DATA ORDERED RY MAGHITUDE OF ERROR TERM 


TERS WILL FRE DISPLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 


FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 


EX CHARACTER ARRAY MANIPULATIONS, ) 


AM GOAL IS TO DISPLAY OBVIOUS PATTERNS THAT MAY THEN RFE 
MORE CLOSELY USING THE 'CURRIC' OR 'COURSE' FUNCTIONS, 


COURSES MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 


MATRIXEMATRIXX(PFMATRIX)P “1 1 1111 
e'IF TOU WANT ALL COURSES DISPLAYED, ENTER 1! 
¢ (OTHERWISE, ENTER Q TO SELECT SOME SUESET OF THE COURSES: 


4JUMP1x10 
¢' THE FOLL 


OWING COURSES ARE AVAILABLE FOR SELECTION 3! 


° 


eKeEXCAMECUNIQUE MATRIXCI3Z]] 
¢'EMTER THE COURSE NUMBERS YOU WANT, ENTER % TO GET THEM ALL,'! 


¢' CHOTE 3 
Yerrureg 
4+JUMPD 
JUMPLS Ver Xe 
JUMPDS$ Ie] 
CRSLOOPi{¢' 
9Ox1I>7T 
(¢' COURSE 


YOU MUST CHOOSE MORE THAN OME COURSE)' 


KCAXEUNTQUE MATRIX(C33)) 


> 'Jsee4C IT] 


ZeECXCIJ=HMATRINCS3ZI)V/CLJ] MATRIX 


*Z ORDER 6 
¢' t 

I¢eI+] 

¢' ' 
4a+CRSLOOP 

7 


Figure A.3 ‘COURSEP! Program 
DD 


ee 
oe ee 


INEU FROME 


CeCURSE 305 


3601 3602 


* see COURSE listing for details 
of the pRompts 


———— 


| 
| 


Figure A.4 Input Syntax ;: COURSE 
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COURSE ; 
“83.4 
“82.4 
“82.2 


~83.2 
“84.2 


| 

“83.4 

~B4.2 

“83.4 

Tos.2 

“84.2 

“83.4 

“84.2 

“83.2 

GROUP 

365.0 

374.0 

: 325-0 
OVERALL; 


COURSE 3; 


~B82.1 
iz PAPER 
"S33 
“83.1 
“84.1 
“84.3 


“83.3 


e333 


GROUP 
374.0 
saue0 


684.0 


OVERALL: 


Figure A.5 


_ Output fron zap 
Figure A.4 =; COURSE Progran. 


3601 


3602 


0 3601 14 2 ey 
0 3601 14 aL a. 
0 3601 14 8 6 
0 3601 14 16 re 
0 3601 ad 26 aa 
365 3601 3 ak “8 
365 3601 yu 0 2 
374 3601 2 <I = 
374 3601 4 bs] me 
374 3601 = 4 ei 
329 3601 2 7 Pl 
323 3601 =O 22 ge 
20 3601 10 1 ~1 
MEAN VARTANCE ad (OF ERROR TERM) 
“3.0 30.0 2-0 
rey, 1.3 3-9 
Te3 4.3 3.0 
“Ana §1050 8.0 
0 3402 7 pea ~4 
0 3602 135 14 1 
0 34602 15 16 ak 
0 3602 7 7 0 
0 3402 LZ ve 0 
0 3602 14 14 0 
374 3402 zs 3 ee 
374 3602 2 4 72 
374 3602 E at 0 
374 3602 o 3 0 
323 3602 12 10 a 
vaso 3602 é é 0 
Jad 3602 11 11 0 
veo 3602 7 7 0 
686 3602 at 0 1 
MEAN VARIANCE N (OF ERROR TERM) 
“1.0 1.3 4.0 
=) 1.0 4.0 
1.0 1.0 1.0 


2) J) 


ut Sequence shown in 





Et 4 

C 2a 

C3] 

C4] 

CSd 

C6] 

C7 

C8] 

C9] 

C10] 
Cita 
Cl24 
C13] 
C14] 
C15] 
C16] 
C17] 
Etsd 
E193 
C20] 
Eat 
C22 

C2354 
C24] 
C29] 
C26] 
eal ge | 
C28] 
C271 
C30] 
ton] 
E32] 
C331] 
C34] 
C351 
C36] 
C37] 
C33] 
ES? 4 
C40] 
C41] 
C42] 
C43] 
C44] 
C45] 
C46] 
C47] 
C48] 
C49] 
60] 
C311] 
boc 
Cage 
C54] 
Egos 
C56] 
CS7] 
Ese 
ES? 


9COURSE[LQ]9 


9 COURSE MATRIKSXZLIVFSIMOHVOZISKIFVIPIMAZSVASWSATRINAZSNCINS SNES IND; AE;SPE 


THIS FUNCTION, GIVEN AN INPUT MATRIK IN THE FOLLOWING FORM 3 


° 


QTR CURRTIC CSE acTUAL PREDICTED (ACTUAL—~PREDICTED) 


WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 
FOLLOWING MANNER ° 


e 
COURSE 
QUARTER 
QUARTER 
QUARTER 
QUARTER 


DATA CURRICULUM 
DATA OR CURRICULUM 
DATA 


DATA 
DATA 


hor 


° 
DATA CURRICULUM N DATA 
MATRIX OF GROUP MEANS, VARIANCES AND WN 


OVERALL MEAN, VARIANCE AND HW 


& G1 he 


A MATRIX OF ACCURACY, PREDICTIVE AND HO ERROR PERCENTAGES IS 
AVAILABLE IF DESIRED, 


(ALL QUARTERS WILL BE DISPLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 


FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 


TO COMPLEX CHARACTER ARRAY MANIPULATIONS, ) 


RRRRRkRKRk NOTE $ THE MEANS AND VARIANCES COMPUTED APPLY RRRARKRER 


RRRRARKR ONLY TO THE INDIVIDUAL CURRICULA 1.555. RRRRRRRR 
RRERRARARRR ANY SUMMARY DATA (ACOLUMN 3D ENTRY = Q) RERKERAKR 
RRARARARKRER IS COMPRESSED OUT OF THE COMPUTATION, RRRERARKR 
RRRARARRAR THIS CAN BE CHANGED BY REPLACING THE #4 RRRARRRRR 
RRRRARKERR SIGN IN THE LOOPS (WHERE <= OR W (352] IS RRRARRRKR 
RRRARAARAR COMPRESSING COLUMN § OR WHERE X2 IS RERRERARR 
RRRRKRKEAR COMPRESSING THE OUTPUT) WITH ‘=' , THISRRRaxxaane 

RRARARARAR FUNCTION WILL THEN DISPLAY ONLY THE RRRRKRKR 


RRRRAAKR SUMMARY DATA, RRRKRAER 


COURSES MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 


>wp»owoveonvnvovvneavnvnvanrnrmwmrwrnarmearrnvxavnanxnvnxavnnnvnvnnvanrrer” 


MATRIX&EMATRIXX( PMATRIX)P “1 1111Liii1 

¢'IF TOU WANT ALL COURSES DISPLAYED, ENTER {' 

¢' OTHERWISE, ENTER Q TO SELECT SOME SUBSET OF THE COURSES' 
»9JUMP1x10 a 

¢' THE FOLLOWING COURSES ARE AVAILABLE FOR SELECTION 3' 
PTNEATAKEUNIQUE MATRIXEL,53])) 

¢'ENTER THE COURSE NUMBERS YOU WANT, ENTER K TO GET THEM ALL,' 

e' (NOTE $ TOU MUST CHOOSE MORE THAN ONE COURSE) ' 

Yerxeg 

2JUMPD 
JUMP TS VTepKeKALAKEUNIQUE MATRIKE 53] )] 

JUMPDS Tet 

¢'THE DATA MAY BE DISPLAYED (TO LOOK AT PATTERNS) IN GROUPINGS BY' 
¢'EITHER QUARTER OR CURRIC, FOR QUARTER GROUPINGS, ENTER 1 5 FOR' 
¢'CURRIC GROUPINGS, ENTER QO! 

Veg 

¢' ACCURACY ERROR, PREDICTIVE ERROR, AND HO ERROR PERCENTAGES ARE' 


r' AVAILABLE, ENTER Y IF DESIRED, N OTHERWISE! 
INDef] 


CRSLOOP?t¢-' | 
~Ox\,I>Y 
(¢' COURSE 3 ')yeX(T] 
ZTeECKXCIJ=MATRPINC'3ZI)/CL] MATRIX 





Figure A.6 *COURSE' Program 
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| [60] MCeMEAN(=052340)/2056] 

C461] VCevarra(=[32340)/2656] 

C62]: NCepr (2052140)/=056] 

C463] vet 

C64] 2NEXKTx .V#1 

C65] @TRe10x(=-2051))-L-2051]3 

[46]  70uT 

C67] NEXT! QAQTReZ[ 32 
C468] OUTLYDePXZEX2ILCSAXDEUNIQUE ATR] 

C469] NSFAECPECNGEMREVGE 1 O 
C702 QTRLOOP ? END) I> D 

| C71] We((K2C4I=QTR)/C1J =) ORDER 6 
C72] MQeMQyMEAN(WL52140)/ML54] 
C73] VQevaQ,VARTA( WE 52140)/WC3 6] 
[74] NG@eNQ,NEep (WO 52340) /4E63 6] 
C75] 2JUMPZx ,INDs'N! 
C76] PEGPE, (+/(4052340)/(40341=0) VWO351=0)=NE 
C77] AEC OE, (4+/( 405 2940)/ (4054140) 404055140) 4405 46140) SNE 
C78] NSENS, (+/(W052340)/(4054140)4( 403514024405 6150) =NE 
C79] JUMPIi ¢u 
C80] -¢' | 
C81] yeve1 
C82] »9@AQTRLOOP 
C83] END? lele+1 
C84] ¢' GROUP MEQN VARIANCE N (OF ERROR TERM) ' 
C8S] 7 1 #(%240)/0£13¢(8%2700.5] M@)y»C2] VR@),C2] NR 


¢' i 

(e' OVERALL: ')y 6 1 eMC, VCANC 
e¢' ' 

SIJUMP 4x, INDS'HN! 

¢! ERROR PERCENTAGES! 

e' GROUP PRE ace NONE a 
7 2 #€%240)/0113¢6¢(08%2,00.395] PE 
¢' 1 
JUMPASe' ! 

»CRSLOOP 

9 


»»,C2] 2&),C2] NS),C27] NR 








Figure A.6 (continued) 
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INPUT PROMED 


COURSE OS 


3601 3602 


* see COURSE listing for details 
of the prompts 


a gs Ym Css Lem SY seca pI a I sy cm fame 


| 


Figure A.7 Input Syntax : COURSE 


6 0 





COURSE ;: 


Aa 


mia Se 
"OAa2 
~84.2 
BES. 2 
'ac..2 
~ 84.2 
~84.2 


~ 83.4 
~ 83.4 


-B82.4 
“83.4 
~83.4 

GROUF 

20 

4.0 
OVERALL: 


COURSE : 
nea 
ee Sut 
woo. 
ooo 
~B84.1 
“84.1 
“84.1 


BS Sic 
~84.3 
~84.3 
~B2.3 
roo. 3s 
83.3 
= rae 
“84.3 


GROUP 


1.0 
3+9 


OVERALL? 





Figure A.8 


3601 
0 3601 14 
0 3401 14 
3465 3601 2 
525 3601 2 
374 3601 4 
525 3601 10 
0 3601 25 
374 3601 3 
0 3601 14 
365 3601 3 
0 3601 14 
374 3601 x 
523 3601 9 
ME AM VARIANCE MM 
~.6 Al 50 
“3.0 25.0 3.0 
SREP O59 Fake 
3602 
0 3402 7 
0 3602 7 
374 3602 1 
525 3402 6 
0 3602 12 
374 3602 3 
525 3602 9 
374 3602 3 
374 3602 2 
Dae 3602 12 
0 3602 iS 
0 3602 Ls 
586 3602 1 
525 3602 11 
0 3602 14 
MEAN VARIANCE N 
0 0 4.0 
m2 3.2 550 
7" M.S 86° 910 


Output fron aa eae Sequence shown in 


Figure A.7 ; E Progran. 


COUR 


[= 


tJ 


hI 


> OoeY Chr Oa W 


hI es PO 
Nake e GW 


s 


foi t t 
ree fo hho Oo 


i 
rd oO NI a .o 


(OF ERROR TERM) 


— 


— 
OWNMNOoORNE 


rs) 
4 
10 
14 
16 
0 
lel 
14 


1 
oOo OO 00 > 


OoorrY Ff hs bh 


(OF ERROR TERM) 


i 


4. Curricula Related Work 


ek ed = =—_ 


In looking at the curricula information, we want to 
find out just how good the figure ‘twelve' is for the 
expected input from the Antiwhale Warfare curriculum. Since 
curriculum 374 will be administering the progran, we will 
use tke data from 374 as an approximation of curriculun 


449.5's performance. Again, the variable 'OS' contains the 


information we want. The "CURRICP" program most’ easily 
shows this data. Through proper input, we can call up the 
historical performance of curriculum 374. The.» ‘CUREI Gas 


program provides all the prompts necessary (see figure A.9). 
Since we are only interested in one curriculum but two are 
reguired for proper frogram execution, a dummy curriculun 
(333) 1s requested. The program output is shown in figure 
A.10 See figure A.11 for a program listing. 

As we can see from figure A.10, curriculum 374 
routinely appears to overestimate the number of students 
expected for any given course. To show more adequately 
patterns of concern in the data (such as performance for 
0S3601 and 0OS3602), we can look at the ‘'CURRIC! prcgran 
sorted Ey course. This will break out the predictions to 
retter show how well curriculum 374 performs for individual 
courses. Figure A.12 shows the inputs, figure A.13 the 
outputs, and figure A.14 the program itself. 

As we can now see from the course data shown in 
figure’ Aa is- curriculum 374 generally overstates the 
expected number of students attending courses 053601 and, to 
a lesser extent, 0OS2602. We can't observe any possible 
patterns by quarter due to the limited number of chserva- 
tions. A brief look at the error percentage table shows no 
predictive errors for either course (they're taught or not 


taught when they say so - no 'zerost in the actual or 
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predicted columns), and an improvement in accuracy from 3601 
to 3602 (0% error free to 50% error free respectively). 

In summary, through use of the curricula related 
programs we have found that a slight overestimation of 
students expected is usual for these courses. This trend nay 
or may not hold for increased class sizes {of, say, twelve 
students vice three or four). Therefor, combined with the 
‘information we previcusly obtained from the course related 
programs, we conclude that new sections will probably not be 


reguired. 


| 


| INEUT PROMET 
CURRICP OS 
* 
0 
ae 


* see COURSEP listing for details 


| 
l 
| 
| 
| 374 363 
| 
of the prompts 


ona 


Figure A.9 Input Syntax : CURRICP 
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rrr ncaa a ES HH TT === —S—S*C~—S*=~C~—NEté=“‘“‘“ C‘s‘OSC;O”;#;#~‘“‘O‘O™O* 


CURRICULUM $ 374 
~83.4 374 2101 0 a) ~6 
~83.4 374 2103 2 a) ~4 
Bisic & 374 2105 1 4 ae 
~83.3 374 3604 i 4 = 
“83.4 374 3601 2 S >3 
Boe.o 374 3602 S oF 72 | 
“83.4 374 3603 A a) ne 
“84.1 374 3604 a 4 a2 
~ 84.3 374 5602 Ps 4 2 | 
7SS.2 374 3601 4 5 ae 
363.2 374 3604 0 if <b 
“84.1 374 2103 0 1 = 
~84.1 374 3401 0 il ean 
~84.2 374 2103 2 3 age | 
~“B4.2 374 3601 3 “4 me 
“84.3 374 3604 2 3 a: 
“83.21 374 3602 1 1 0 
S8Sel 374 5604 & 4 0 
~ 84.1 374 3602 ee 3 0 
~“B4.1 374 3603 1 1 0 
CURRICULUM §$ 333 


| lS Se 


Figure A.10 Outrut from Input Sequence shown in 
Figure A.9 ; CURRICP Program. 
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C1ij 

Bad 

bo 

C4] 

CS] 

bod 

7 

C83 

C9] 

C10] 
eat ae 
Eta) 
ELos 
C14] 
bios 
E162 
Elza 
C18) 
C19] 
Eaod 
E2id 
E22 

C23] 
C24] 
aad 
E262 
C27] 
C28] 
E29 J 
C30] 
Est] 
Eo.) 
C33] 
C34] 
C35] 
C36] 
E373 
C38) 
C39] 
C40] 
C41) 


9CURRICP(Q]YV 


9 CURRICF MATRIX;X3I3732 


THIS FUNCTION, GIVEM AN INPUT MATRIX IN THE FOLLOWING FORM 3 


QTR CURRIC cseE acTualL PREDICTED (ACTUAL-PREDICTED) 


WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 
FOLLOWING MANNER 3 


e 


CURRICULUM 
DATA ORDERED BY MAGNITUDE OF ERROR TERM 


(ALL QUARTERS WILL RE DISPLAYET AS NEGATIVE NUMBERS, THIS ALLOWS 
FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 
TO COMPLEX CHARACTER ARRAY MANIPULATIONS, ) 


THE GOAL OF THIS PROGRAM IS TO DISPLAY OBVIOUS PATTERNS THAT 
MAY THEN BE LOOKED AT MORE CLOSELY USING THE 'COURSE' OR 'CURRIC' 
FUNCTIONS, CURRICULA MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 


>rrrermerrerermerererererere rR » > ® 


MATRIXEMATRIXX(PMATRIX)P “1 14141411 { 
e'IF YOU WANT ALL CURRICULA DISPLAYED, ENTER 1' | 


¢ ‘OTHERWISE, ENTER QO TO SELECT SOME SUBSET OF THE CURRICULA! 
+JUMP1x10 
¢' THE FOLLOWING CURRICULA ARE AVAILABLE FOR SELECTION 3¢' 
PASCAL AXEUNTQUE MATRIXKE 52733 
e'ENTER THE CURRIC NUMBERS YOU WANT, ENTER X TO GET THEM ALL,' 
¢' (NOTE $ YOU MUST ENTER MORE THAN ONE CURRICULUM) ' 
Yepreg 
2JUMPOD 
JUMP YTS TEPXEXCAXEUNIQUE MATFRIXKX( 5233 
JUMPDP$ Tey 
CRCLOOPS3¢'* ! 
20x. 
(Ce 'CURRICULUM § ')y,@#XCI] 
Se(CALCIJHEMATRIKC323)/C13 MATRIX 
@= ORDEF 6 
Tele 


xa~CRCLOOP 
v 


* 


Pigure A.11 'CURRICP* Progran 
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374 333 


| 

| 

| 

| * see COURSE listing for details 
of the prompts 


Figure A.12 Input Syntax 
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CURRIC 


[Ss a ee Se SR ete a es ND a ee) ae Ci ee yee ee ee Se 


EY 


CURRICULUM +: 374 
“33.4 374 eroL 
| “83.4 374 =2LOS 
83.2 374 Beds (018 
“84.1 374 2103 
“84.2 374 2103 
“84.1 374 3401 
“83.4 374 3601 
= EP 374 3601 
“84.2 374 3601 
“83.3 374 3602 
“84.3 374 3602 
“83.1 374 3602 
“84.1 374 3602 
“83.4 374 3603 
“84.1 374 3603 
“83.3 374 3604 
“84.1 374 3604 
“83.2 374 3604 
“84.3 374 3604 
“83.1 374 3604 
GROUP MEAN VARIANCE 4A 
2101.0 “$0 1.90 1.9 
2103.0 wee BaRIE 4.0 
3401.0 1.0 1.0 1.90 
3601.0 n> ers ara 3.90 
3602.0 “1.0 Les 4.90 
: 3603.0 “1.0 209 2.0 
3604.0 “1.4 io 3-90 
OVERALL 3; les Zea 2020 
ERROR PERCENTAGES 
GROUP PRE acc NONE 
2101.00 1.00 00 ~ 90 
2103.00 vas AP fea «90 
3401.00 1.00 » 00 90 
3601.00 00 1.00 ~90 
3602.00 00 »90 »30 
3603.00 »00 ~ 30 » 50 
3604.00 » 20 60 ~20 
CURRICULUM 3: 333 
GROUP MEAN VARIANCE WN 
OVERALL °¢ 14:0 0 raze) 
ERROR PERCENTAGES 
GROUP PPE acc NQUE 


Figure A.13 


Figumer aA. 12; 
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0) 4 =a 
= 5 = 4 
i 4 as 
0 i =i 
2 3 =a 
0 1 a 
2 5 =% 
4 a) =1 
3 4 ToL 
3 5 22 
A 4 =o 
iL yl 0 
3 R 0 

ante 
i 1 0 
1 4 ae 
e + 7a 
0 1 = 4 
Ai, 3 ogi 
4 4 0 


(OF ERROR TERM) 


1.00 
4.00 
1.00 
3.00 
4.00 
2.00 
3-900 


(OF ERROR TERM) 


Ceggees eee ES) OS pee 
a a i OS OED 

ea lc ET yeep men aR SE I et lS a te CED 

ee 

a a ree 

np pea 


Outrut from Input Sequence shown in 
CURRIC Progran. 


eee 


VCURRICCQ}V 


9 CURRIC MATRIXSXS ISTE ZEMOHVCZIEXDSTISMASVASWSASNSSNESINDS AESPESV3NCZHG 


C26] MATRIXEMATRIXX(CPMATRIX)P “41 1111 21 

C27] ¢'IF YOU WAMT ALL CURRICULA DISPLAYED, ENTER 1! 

C28] e' OTHERWISE, ENTER QO TO SELECT SOME SUBSET OF THE CURRICULA' 

C29] 3JUMP1x10 

C30} -¢' THE FOLLOWING CURRICULA ARE AVAILABLE FOR SELECTION !3' 

CZiy eXexXCaxeuNIQUE MATRIXL52)] 

C32] -¢'ENXTER THE CURRIC NUMBERS YOU WANT, ENTER X TO GET THEM ALL,' 

C33] e' CHOTE §$ YOU MUST ENTER MORE THAN ONE CURRICULUM) ' 

C34] vrerxeg 
E35] +JUMPOD ; 
C36] YUMPLSYeEPKEKECAXCUNIQUE MATRIXE 32) ) 

C37] YuMP2{re1 

C38] -¢'THE DATA MAY BE NISPLAYEDN (TO LOOK AT PATTERNS) IN GROUPINGS EY? 
C39} -¢' EITHER QUARTER OR COURSE, FOR QUARTER GROUPINGS, ENTER 14 35; FOR' 
C40] e' COURSE GROUPINGS, ENTER 2,' 

C413 veo 

C42] -¢'ACCURACT ERROR, PREDICTIVE ERROR AND HO ERROR PERCENTAGE MATRIXN?? 
C43] e'ENTER tr OR WN? 


CijJ A 

eal a THIS FUNCTION, GIVEN AN INPUT MATRI“% IN THE FOLLOWING FORM ¢ 

C3] A 

C4] A QTR CURRIC CSE ACTUAL PFENDICTED (ACTUAL-FREDICTED) 

C5] A 

Fe] aA WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 

C74 a FOLLOWING MANNER $3 

C8) A 
Eo] 4& CURRICULUM ‘ 

C10] a QUARTER 1 DNATA COURSE 1 DATA 

Ciliin . QUARTER 2 NATA OR COURSE 2 DATA 

F414) in QUARTER 3% DATA : 

riz) -« QUARTER 4 DATA COURSE N DATA 

C14] a MATRIX OF GROUP MEANS, VARIANCES AND N 

CiSl a OVERALL MEAN, VARIANCE AND N 

C16] a 

C17] an @® MATRIX OF ACCURACY, PREDICTIVE AND NO ERROR PERCENTAGES IS 
Ci8] pa AVAILABLE IF DESIRED, 

C19] a 

C20] a (ALL QUARTERS WILL BE DISPLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 
C21] an FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 
C22] a TO COMPLEX CHARACTER ARRAY MANIPULATIONS, ) 

C23] a 

C24) a CURRICULA MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 

253 | 
7S eS) A 


C45] CRCLOOP$,¢:' ! 


C46] 70x1tdyY 
C47] (¢' CURRICULUM $ '),9X[T] 

C48] Ze(CXNECIJHMATRIXCL523))/C1)] MATRIX 
C49) MCeEMEANM 2536] 

C50) VCeEVAFRTIA 2£5;6] 

Gold Heereats od 

ES2j Jel 

E532 SMNEXTX ,V=3 

C54] S410x(-Z2034))-L-Z2054) 

C55]  »0urT 

C54] NEXTIACZCj5V] 

C57] OUTS YDePXDIeEXDLAXQEUNIQUE A] 
£58] MStAECPEcHaemMQevaeyiO 

CS9] S&TRLOOP § +EMDy 1 I)? 


lp in, te AR i eR RR eal ey pr A cy I sy Le ey ern a alae SERN IRE a iI a IMI sa a fa 





Figure A.14 *"CURRIC* Progran 
6 8 


C60] 
G61 1 
C62] 
Ga3 1 
C44) 
C65] 
C466] 
bows 
C68] 
Bo? 3 
C70] 
Evia 
G72] 
ES 
C74] 
G7o2 
E7o 
C77) 
b78) 
C79) 
C80] 
Beid 
ES2] 
C83] 
C84) 


i jac i gE, fay AARIID Pa i a a RIE gt RO TED MR, eA Gt ees A iat, SEE le ca AD SD, 


We( (XOCIIJ=A)/CE1] 2) ORDER & 

MQeEMQ,MEAH WO5 6] 

VQEVQA,VARIA WL36] 

HAENA, HE eEPEWL 5 6] 

SIJUMPTBXLIND='N! 
PEGCFE,(+/(4C 54 7=0) vWC557=0) NE 

APECAE, (+/(40 54740) 4040 551740)440 7 6140) HVE 
MSENS, (+/(C4C 5S 49404 (40 55740) 4405 6I7=0)HHE 


JUMP? gw 


t ¢ 


¢ 
Jed+] 
3ATRLOOP 


END$Iel+] 


2! GROUP MEAN VARIANCE NW (OF ERROR TERM)! 
7 1 #0 ¢€8%2,00.,5] 4&@),C2] VA),C2] NA 

¢' t 

(e' OVERALL $ ')y 6 1 eMC yVCynec 

¢' t 

SIUMP AX LIND='H'! 

¢! ERROR PERCEHTAGES'! 

¢' GROUP PRE ace HOME HS 

7 2 ¢606¢68%2500.5) PE,C2) AEV,C2] NS)5,C2] NG 

¢' 4 


JUMP 4i¢! ! 


4CRCLOOP 
7 


e 


Figure A.14 (continued) 
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Bieest gon geo epee gee Pee ee ee 


D. DATA COLLECTION AND MAINTENANCE 


To provide the matrix of historical data to be displayed 
and analyzed, two sources are used. The prediction data 
comes from the course loading forecast delivered to the 
department chairmen (see figure A.15). The easiest way to 
compile that data is to obtain a copy of the individual 
course predictions and, on the forecast itself, change the 
letter codes found in the "S/D/T" column into curricula 


humbers (using the coding list contained in appendix B) with 


associated totals. For example, the totals you would write 
in from figure A.15 fcr%S3602 "adresse, soe 3, anda 
(EeGd lcs eee. The cther document reguired is the course 


grade sheet held by the registrar (see figure A.16). This 
information is subject to the privacy act and requires a 
written reguest for permission from the department chairman 
involved to the office of the registrar for access tc the 
files. The records themselves are kept according to 
academic year. This notation is confusing; to prevent prob- 
lems, ask for something like the '83-84 school year data. 
These records are not allowed to leave the office, and at 
last check were not available on the computer. The data 


must ke recorded by hand. A convenient format is: 
Quarter Course Section Total# #Attending/Curricula 


Once the above listed data have been collected and are 
in frent of you and a computer terminal, a CMS file as shown 
in figure A.17 can be created by combining the informaticn. 
The columns correspond to the guarter, curriculum, course, 
actual and predicted number of students. The CMS file thus 
created can be transferred to APL and named, for instance, 


CSNEXT through the fcllowing command sequence: 
{get into APL) 


) LOAD 990 CMSIO 
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) KSID HOLD 
OSNEXT <= CMSREAD 


(follow prompts to read your CMS file) 


) ERASE CMSIO 
) SAVE 


NOTES : 1) The APL variable '"OSNEXT' now exists 
On your disk in the APL workspace ‘HOLD! 


2) The *<=" character shown corresponds to 


the left-pointing assignment arrow in AFL 


To add the (Actual minus Predicted) term to the matrix, 


enter: 
Sei = SOSNEXI,( 2] OSNEXTE;>4 |] — OSNEXT[ 55} 


To add this matrix tc the variable OS (assuming that OS is 


in the workspace), enter: 
CS <= 0S,[1] OSNEXT 


The entire updating frocedure should take about one full 


workday for each year of data. 
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Ee OTHER AVAILABLE PROGRAMS AND DATA 


A ccemplete listing of the functions available in the 
"OSDATA' workspace is Shown in table VII. The COURSEP, 
SOURSE, CURRICP and CURRIC programs have been previously 
demonstrated. The orly non-trivial function remaining to be 
shown is the 'WILCOXSK' program (see figure A.18). AS shown 
in the figure, this program is based on the Wilcoxian Signed 
Rank Testi! detailed in [Ref. 5 pp 280-288 }. 

A self-descriptive listing cf the variables contained in 
the wcerkspace 1S Shown in table VIII . The latest data pres- 
ently available is the spring of 1984 (academic year 83-84) 


figures. 


llFor a complete description of the _vaLrious BONO Bhai 
that Recrui ore os see [Ref. 5 PP 280-288 }. In the bod 
e 


of the esis, the magnitudes of forecaSting errors (y 

were tested against he magnitudes of prediction errors 
derived from course. matrices ({x). The ngenests tested 
(with rejection the desired result) was that the 'x! values 


tend to be equal to or larger than the Oe values. This was 
one of six possible interpretations of the null hypothesis. 
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ne ee ee eT ee 


| 


TABLE VII 


Functions Available 


on Syntax Description 
wk kk KK KK He ke ok KK Sek otek ok ke ke ok ok kok ok ok kk ok KK tk 


CCURSEF COURSEP mtx (previously shown) 


GCURSE COURSE mtx (previously shown) 
CURRICP CURRICP mtx (previously shown) 
CURRIE CURRIC mtx (previously shown) 


WIICOXSR y WILCOXSR x Shows the Wilcoxian Signed 
Rank 1 Cs Gag selena 
description in text) 


ORDER A ORDER 8&8 Orders rows of matrix A 
ac ceraad to decreasin 
nitude of column B, retu 
the ordered matrix 


UNIQUE UNIQUE A Returns the unique elements 
of the vector or matrix A in 
ascending order 


Mag- 
rning 


MEAN MEAN X Returns the arithmetic mean 
of vector X 
VARIA VARIA X Returns the unbiased estimate 


of the variance of vector X 
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Figure A.18 
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9WILCOXSR(QV9 


9 YT WILCOXSR X;D;T5R 


THIS FUNCTION PROVIDES THE WILCOXIAH SIGHED RANK TEST 
STATISTIC (FROM CONOVER PG, 280), INPUTS XK AMD YT AKE 
DATA VECTORS OF EQUAL LENGTH 


De(D#0)/DeEeY—-x 

Re (RANK(D)xD+4D 
Te C#/R) SC 4+/R22) 20 
(¢'TEST sTaTrsTre 
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* 


Wilcoxian Signed Rank 


ee 


ee eee se ee ee » 


es) 
= ')yeT 


yey 





Test Progran 


Mie eg ae EE a Ee anime SS ee patie Gi aat Spats EE tel 


Variables Available 


TABLE Vill 


VARTAELE SHAPE EPESCRIPT Ugh 
SR Ko ok kok Kk ek 
OV SRE 69x6 AY {Academic year) 81-82 data; 
includes SUR De eo ere tre 2 
(no data for individual curricula) 
0S83 2555.6 AY 82-83 data; complete listing 


OSTRK€3 140x8 AY 82-83 non-entry course data. The 
additional columns contained in this 
matrix (columns 6 and 8) correspcend 
tc an *adgusted spre dtectrren sand au 
"ad juSted”™ CErOr™ Tern dchived trom 
the course matrices. This array was 
used to check the feasability of 
rae adjustments based solely on 
the 'frack' a student takes using the 
Wilcoxian Signed Rank Test previously 
mentioned. 


Ooséy 22 8x6 AY 83-84 data; only qvrs 1,2 ance 


cS 552x6 Ccmbination of all of above 
{without repetition) 


| 

| 

| OA83 nes x 9 This matrix is set up in a different 
format from those previously dis- 
cussed. The nine columns represent 
Cuarter, OA course number, Total 
attending, USN attending, USA 
attending, USMC att eo ee Others 
attending, Total predicted, and Total 
attending minus’ total predicted 
respectively. The original. ntent, 
dropped due to time constraints, was 

te use these pee tOvard #1 On 

elective scheduling by looking at 

expected number of upcoming free 

electives by service through use of 

the OA course matrix. 


[yy pe es GIs) Se eto i Ge ae ee ed A i eR fee Ge a Oe ED CE) ee Ss ee es ee 
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SCHEDULING CODES 


This appendix ccntains an alphabetical listing of the 
specialty codes used in the scheduling process as of August 
1984. This listing cnly uses the first two letters of any 
individual coding because the remaining letters specify 
€ither the degree awarded or the degree and month of input, 
neither cf which change the curriculum associated. Using 
these codes, numbers of students predicted by any given 
curriculum for any given course can be taken from the indi- 
vidual course forecast promulgated by the programming 
office. Code 0301 at extension Zi Seat the Naval 
Postgraduate School can be contacted for updated 
information. 

Specialty 
Curricula Degree 


Numbers Codes Frogram 
Sk tok kkk ak ak tok ok kit kok kk kok kok ikki ok kok kok kk kok kok ok kk kkk 


610 Nea Aepow nga, lnel Engr sci 
A= 
611 a Aeroelectronics 
368 oS== Computor Science 
600 Dé=r Electrical Engineering (Comms) 


Note: DCTR is curriculum 460 


90 ea Electrical Engineering ; various 
2) sub-specialties. Note: EETR is 
EG-- curriculum 460 
Ek = 

380 Ci Math option of advanced science 


TS 


Specialty 
Curricula Degree 


Numbers Codes Frogram 
fokoK oR ak kkk kok kok kkk ok kkk kkk gk KK RK KK RK Kk x 


381 GP-= Fhysics option of adv. science 
620 HG== Communications Management 

Hlle> 
525 iXS= Antisubmarine Warfare Note: IETR 


is curriculum 460 


365 0) Command, Control and Communica- 
Is tions ; various sub-specialties 
Jo=— Note: JCTR and JXTR are from 
JBe= curricula 460 
Jo== 

De) 5 Kin Electronic Warfare Systems Tech- 


nology Note: KETR is curric 460 


817 ME--  ~+Admin Sci - Econ, USMC 

Ba 7 ie Admin Sci - Financial mgt 

857 gl Admin Sci - Human Resource nagt 
819 oa Admin Sci - Inventory 

817 Wh Admin Sci - International Stu. 
827 Nyy Admin Sci - Material 

817 fO-— Admin Sci - Ops Research, Army 
847 ie =— Admin Sci - Personnel 

815 Re Admin Sci - Procurement 

817 MS-- Admin Sci - MGMT Sci, Coast Guard 
814 w= Adnin S@®W—* Transportation 
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Specialty 
Curricula Degree 


Numbers Codes Frogram 


Fe He SK ee ee ee ee a ee Ke oe Re oe ko 2 ok ee ok Ko ok eo ok Kk ek kK ok ok ok ok & 


813 


817 


Se) 


447 


440 


567 


360 


460 


So) 


a7 | 


366 


eye 


NVe= 


WD Goatees 


1 era 
NES 
Neca 
Nia 
1 


Ok== 


OTS tes 


ei== 


RA-- 


kC=< 


RG-- 
RN-- 
RS-- 
RP-- 


Sp leg 


oO 


UxX== 


Admin Sci - Material Movement 
Admin Sci - Civ. Federal Program 


Mechanical Engineering 
ME via 460 
ME via 460 
ME via 460 
ME via 460 


Hydrographic Science 
Cceanography Note: OCTR is 460 
Information Systems (from CS) 
Crs Analysis, Army 

Page SCim@Lor OpsmAnalysis 


Cps Analysis, General 
Navy 


Sup oly 
PHD program 


Space Systems Engineering 


Space Systems Operations Note: 


SXTR and SOTR are curriculun 460 


Underwater Acoustics Note: UETR 


1s curriculum 460 
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Specialty 
Curricula Degree 


Numbers Codes Frogram 
SRG a Ro kkk kok kak kok kok akc oka keke akc ae ot akc ak 


3310 WA-- Weapons Systems Technology 
VC=— 
Wises 
Us 
S yo WN-- Weapons Systems Science 
53 0 iP-— Weapons Systems Technology 
531 oe Weapons Systems Science 
Sez ae Meteorology 
373 XS== Met. and Oceanography 
374 Y= Ailr-Oceam Tactical Ene Support 


Note: XOTR' and XXTR are 460 


683 ele NSAv— USSR, Ewsopes MUS N 

682 De NSA - Asia/Pacific; USN 

681 YC-- NSA - Middle East, Africa; USN 
YD-- - S. Asia; USAF, USA 

682 YE-- NSA - ASia/Pacific; USAF, USA 

681 Oe NSA —- Ss AEFI GarmeUSAr, USER 
YG-- = N. AEEFICasmUSAr, USA 
YM-- - Middle East; USAF, USA 

687 a NSA - Nucir/7eing,sUSN 

684 YO-- NSA - Intl/Org/Neg; USN 
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Specialty 
Curricula Degree 
Numbers Codes Frogram 


KK RRR RK KR RR KK RK KKK KKK K KK RKKKKKKRKKKK KKK KK 


686 YP=—> NSAG = esiebat7 Ping. USN 
683 YRS = NSA - USSR/E. Europe 
a5 YS=— NSA - Navy Intell; USN 
683 a= NSA —- USSR/W. Europe 
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APPENDIX C 
DETAILED SUMMARY OF RESULTS 


This appendix ccntains a brief description of the 
results found for each course and curriculum. These results 
are listed in ascending numerical order first for the 
courses then the curricula. Courses or curricula that cause 
problems are flagged with a ‘'P* in front of the identifying 


number. 


Course 
Number Comments 
Stee eS ee eee ee SSS SSS SS SSS SSCS SSS SOS SSS SSS SS OCS SoS LS. 2S. 2 Se 


P 2101 Not taught, only predicted. 


Pez 102 Large magnitude errors, generally underestimates. 
Curriculum 590 causes most of the problems, curric- 


ulum 600 to a lesser extent. No quarterly pattern. 


Po 218 Generally gocd except for curriculum 531 which 
never sends feople but has predicted 40, 27 and 7. 


No significant quarterly pattern. 


P 3001 Prediction errors in third and fourth quarters - 


not predicted but taught. Only curriculum 368. 


3002 Generally gocd. Only taken by curriculum 825 though 


others are predicted. Need more data. 
3003 Generally good. Need more data. 
3004 Generally gccd. 


3005 Generally good. Need more data. 
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Course 
Number 
sek ok ok kK 


3006 


300€ 


301% 


3102 


3103 


3104 


B10 > 


3302 
s}5)(0)p5) 
3401 


3402 


Comments 
-oeee ee eee SES SESSCeSSe SS SOSCSOSSSCOS SSS SSS LS SLO SESS SL £2 2 eS 


Generally gcod except for curriculum 817. Only 
LANGMewarESteand thivay quarters. Overall class 
sizes fairly stable at 63 and 87 respectively 


(three secticns required). 


Definite prediction problem that appears to be 
improving. Curriculum 365 provides most of the 


Ineo ult. 


Curriculum 460 not predicted generally causing a 
Significant underestimation (usually results in 


adding an additional section). More data needed. 


Overall significant overestimation from cumulative 


Sttecarmoneo xk Curricula. Worst in third quarter. 


Overall significant overestimation from cumulative 


effect of 8XX curricula. No guarterly trend. 


Large errors in summary predictions generally due 
GOompmecaIGCE@on Vice aCcWbacy errors. All curricula 
except 570 and 817 experience these errors. First 
quarter predictions look good, underestimates occur 


in the second quarter, overestimates in the third. 


Significant prediction problems. Generally cffered 


only third quarter. 

Not enough data. 

Generally geod. 525 provides majority of input. 
Not enough data. 


Prediction problem, not severe. 
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Course 


Number 
ok ok Kk 


3403 


P 3404 


3601 


3602 


Secs 


3604 


P 3702 


4601 


4602 


4701 


Comments 


TK AK AK AK AK KK IK A KKK A IK i AK KK Ke KK Ke eK eK kK KK KK a 


Generally 


Curricula 


excellent, not enough data. 


365 and 825 provide most of the inputs 


to this course. 365 does an excellent job, 825 a 


poor one. 
Generally 
Generally 
Generally 


Generally 


gGCCis 
excellent. 
excellent. 


good. 


Significant frediction problem. Not taught fiese 


guarter, only predicted. Curriculum 847 provides 


most of the input. 


Generally excellent. 


Generally 


Generally 


excellent, needs more data. 


gocd. 


FR KK I EK OK ae A 2 Re ae eK ae i eo i 2 a 2 a eo oe ee KK eo ek ek 


KK 


END OF COURSE LISTING 7 


Me he kc oe ok i ae KK KK KK KK KKK KK KKK OK KK KKK KKK KKK 
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Giles culun 


Numcker 
KK KKK 


p 360 


BiG. 
36€ 
oon 
P 368 
5) 1) 
3) 7) 2 
P 374 
380 
44Q 
G41 


F 460 


soe. 


50) 


Comments 
we ke Ke ke ake ee ke fe he he he ok ok ok oe ke he 2 oo KK KK ee KK KK KKK KK KK KK OK KK EK 


Summary predictions are innaccurate due to course 


effects. ho quarterly trend seen. 


Seldom predicted for OS courses. Minor effect due 


to small number of inputs. 

Generally excellent, possible problem with 0S3008 
Generally excellent. 

Generally good, need more data in 0S3101. 

Third quarter prediction problem developing. 

Not enough data. 

Not enough data. 

Tends to overestimate. 

Not enough data. 

Not enough data. 

Not enough data. 

Never predicted. 

Generally excellent. 

Problem in 0S3104. 

Significant problems affecting 0S2103 and 0S3104. 
Not enough data. 

Not enough data. 


Possible errors in third guarter, need more data. 
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Curriculum 


Number 
He kK ke ke 


P wood 


5 ou 


600 


610 


620 


683 


686 


687 


B15 


lg Ge 


P 817 


Pp. 87s 


P 827 


iy 121.5) 7 


P 847 


P S57 


399 


- 


Comments 


FO Go a a oak kak kok kok kok ok tok kk kak dk ok kk kak ok ok ak kok tok oo 
Problem in 0S2102 

Looks good, not enough data. 

Generally good. 

Not enough data. 

Not enough data. 

Generally good. 

Not enough data. 

Not enough data. 

Not enough data. 

Generally excellent. 

Generally excellent. 

Generally fair. 

Problem with 0S3006, good otherwise. 

Various prediction problems. 

Problem with OS3404. 

Overestimates for 083102 and 0S3103. 

Problem in OS3102 , ~predterionsernor for OSots. 


Problem with 0O0S3702. Good with 0S3103. Thread 


quarter predictions are poor. 
Routine innaccuracies with no apparent pattern. 


iNSigni& Veant Papuce 
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APPENDIX D 
CONDENSED COURSE MATRICES 


This appendix contains the condensed course matrices for 
all curricula that take OS courses. Curricula 610, 611, 681, 
682, 683, and 684 dc not have any OS courses required and 
are not included. 'xxxxxx' entries in the condensed matrices 
indicate that the students are onboard Fut not taking OS or 
OA designated courses (these entries are not made follcwing 
the final OS or OA course taken to prevent display clutter). 
The entries are based on the printed course matrices as 
publisted as of August, 1984. These matrices were frovided 


by the individual curricular offices. 
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ENTRY / TRACK COURSE DESIGNATIONS 


This appendix contains a complete listing of the OS 
courses divided into ‘Entry and ‘Follow-on' catagories. An 
‘Entry’ course is the first OS course that is taken ina 
given course matrix (for a given curriculum); the first time 
a curriculum can tFLe *seen* by the Operations Analysis 
Department. ‘Follow-cn' courses are those courses that are 
taken in guarters fcllowing the guarter the ‘Entry course 
as taken. All curricula take one of the six 'Entry' courses 
prior to any further OS course work with the exceptions of 
curricula 610, 611, 681, 682, 683 and 684 which have no 


required OS courses. 


EMERY TRACK (Follow-on) 


Coy oe )CURKECULUM  CCURSES, ALI OS 
Rae RR RR tok kk RK 


0S2103 365 3604,3404,3008, 3603,4602 

366 3604,3404,3008, 3603,4602 

374 S601, 3602 (36037 3604 

525 3303,3604,3601,3401,4601,3602 

3) 8)5) hone 

55 3604,3003,3603,3403,4601 
GS31101 567 3004 

620 3005 

686 none 


3 


ENTRY 
COuUkoe 


KKK KK K 


CS3101. 


0S3001 


0S3104 


CS2102 


OS5 102 


CURKRICULG 


eK eK eK OK eK 


687 


825 


368 


Be) 2 


573 


440 


4Qy 


50 


531 


532 


570 


aye) | 


600 


813 


814 


815 


817 


TRACK (EF olvaw—on) 
CCURSES, ALi@e@s 


Fe AK I eK I eo KK KK KKK OK KOK KK KK kK Kk & 
hone 


3404,3002 


none 


none 
none 
none 
none 
none 
none 
none 


none 


3104 
none 


none 


SUS 75006 
on05, 5006 
S05, 50 06 


3103, 3006,3604 
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ENTS Y TFACK (Follow-on) 


BeGroE CURRICULUN CCURSES, -ArEL 0S 
HA a ee ao a a a 


G53 102 Cig 3108 35 O06 
San 2103, 3006 
837 S03, 9006 
847 3 Ge, SO UG OZ 
657 athe as, stow 
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